Engagement is a necessary condition to test audit and feedback design features: results of a pragmatic, factorial, cluster-randomized trial with an embedded process evaluation.

BACKGROUND: While audit & feedback (A&F) is an effective implementation intervention, the design elements which maximize effectiveness are unclear. Partnering with a healthcare quality advisory organization already delivering feedback, we conducted a pragmatic, 2 × 2 factorial, cluster-randomized trial to test the impact of variations in two factors: (A) the benchmark used for comparison and (B) information framing. An embedded process evaluation explored hypothesized mechanisms of effect. METHODS: Eligible physicians worked in nursing homes in Ontario, Canada, and had voluntarily signed up to receive the report. Groups of nursing homes sharing physicians were randomized to (A) physicians' individual prescribing rates compared to top-performing peers (the top quartile) or the provincial median and (B) risk-framed information (reporting the number of patients prescribed high-risk medication) or benefit-framed information (reporting the number of patients not prescribed). We hypothesized that the top quartile comparator and risk-framing would lead to greater practice improvements. The primary outcome was the mean number of central nervous system-active medications per resident per month. Primary analyses compared the four arms at 6 months post-intervention. Factorial analyses were secondary. The process evaluation comprised a follow-up questionnaire and semi-structured interviews. RESULTS: Two hundred sixty-seven physicians (152 clusters) were randomized: 67 to arm 1 (median benchmark, benefit framing), 65 to arm 2 (top quartile benchmark, benefit framing), 75 to arm 3 (median benchmark, risk framing), and 60 to arm 4 (top quartile benchmark, risk framing). There were no significant differences in the primary outcome across arms or for each factor. However, engagement was low (27-31% of physicians across arms downloaded the report). The process evaluation indicated that both factors minimally impacted the proposed mechanisms. However, risk-framed feedback was perceived as more actionable and more compatible with current workflows, whilst a higher target might encourage behaviour change when physicians identified with the comparator. CONCLUSIONS: Risk framing and a top quartile comparator have the potential to achieve change. Further work to establish the strategies most likely to enhance A&F engagement, particularly with physicians who may be most likely to benefit from feedback, is required to support meaningfully addressing intricate research questions concerning the design of A&F. TRIAL REGISTRATION: ClinicalTrials.gov, NCT02979964 . Registered 29 November 2016.

Behavioral Nudges to Improve Audit and Feedback Report Opening Among Antibiotic Prescribers: A Randomized Controlled Trial.

Background: Peer comparison audit and feedback has demonstrated effectiveness in improving antibiotic prescribing practices, but only a minority of prescribers view their reports. We rigorously tested 3 behavioral nudging techniques delivered by email to improve report opening. Methods: We conducted a pragmatic randomized controlled trial among Ontario long-term care prescribers enrolled in an ongoing peer comparison audit and feedback program which includes data on their antibiotic prescribing patterns. Physicians were randomized to 1 of 8 possible sequences of intervention/control allocation to 3 different behavioral email nudges: a social peer comparison nudge (January 2020), a maintenance of professional certification incentive nudge (October 2020), and a prior participation nudge (January 2021). The primary outcome was feedback report opening; the primary analysis pooled the effects of all 3 nudging interventions. Results: The trial included 421 physicians caring for >28 000 residents at 450 facilities. In the pooled analysis, physicians opened only 29.6% of intervention and 23.9% of control reports (odds ratio [OR], 1.51 [95% confidence interval {CI}, 1.10-2.07], P = .011); this difference remained significant after accounting for physician characteristics and clustering (adjusted OR [aOR], 1.74 [95% CI, 1.24-2.45], P = .0014). Of individual nudging techniques, the prior participation nudge was associated with a significant increase in report opening (OR, 1.62 [95% CI, 1.06-2.47], P = .026; aOR, 2.16 [95% CI, 1.33-3.50], P = .0018). In the pooled analysis, nudges were also associated with accessing more report pages (aOR, 1.28 [95% CI, 1.14-1.43], P < .001). Conclusions: Enhanced nudging strategies modestly improved report opening, but more work is needed to optimize physician engagement with audit and feedback. Clinical Trials Registration: NCT04187742.

Effectiveness of confidential reports to physicians on their prescribing of antipsychotic medications in nursing homes.

BACKGROUND: Antipsychotic medication use in nursing homes is associated with potential for harms. In Ontario, Canada, an agency of the provincial government offers nursing home physicians quarterly audit and feedback on their antipsychotic prescribing. We compared the characteristics of physicians who did and did not engage with the intervention, and assessed early changes in prescribing. METHODS: This population-level, retrospective cohort study used linked administrative databases to track prescribing practices in nursing homes pre-intervention (baseline), immediately post-initiative (3 months), and at follow-up (6 months). Exposure variables identified whether a physician signed up to participate (or not) or viewed the feedback following sign up (or not). Differences in the proportion of days that residents received antipsychotic medications at 6 months compared to baseline by exposure(s) were assessed using a linear mixed effects regression analysis to adjust for a range of resident, physician, and nursing home factors. Benzodiazepine and statin prescribing were assessed as a balance and tracer measures, respectively. RESULTS: Of 944 eligible physicians, 210 (22.3%) signed up to recieve the feedback report and 132 (13.9%) viewed their feedback. Physicians who signed up for feedback were more likely to have graduated from a Canadian medical school, work in urban nursing homes, and care for a larger number of residents. The clinical and functional characteristics of residents were similar across physician exposure groups. At 6 months, antipsychotic prescribing had decreased in all exposure groups. Those who viewed their feedback report had a signicantly greater reduction in antipsychotic prescribing than those who did not sign up (0.94% patient-days exposed; 95% CI 0.35 to 1.54%, p = 0.002). Trends in prescribing patterns across exposure groups for benzodiazepines and statins were not statistically significant. INTERPRETATION: Almost a quarter of eligible physicians engaged early in a voluntary audit and feedback intervention related to antipsychotic prescribing in nursing homes. Those who viewed their feedback achieved a small but statistically significant change in prescribing, equivalent to approximately 14,000 fewer days that nursing home residents received antipsychotic medications over 6 months. This study adds to the literature regarding the role of audit and feedback interventions to improve quality of care.

Enhancing Quality Care in Ontario Long-Term Care Homes Through Audit and Feedback for Physicians.

The province of Ontario in Canada is an example of a jurisdiction that is using a specific quality improvement mechanism, known as "audit and feedback," to help clinicians enhance the care they provide to long-term care (LTC) home residents. This program, launched in 2015, is unique in that the reports are provided across an entire jurisdiction. These reports were co-designed with clinicians, including medical directors, scientists, and other stakeholders, and are updated regularly to maintain their relevance to medical practice in LTC. The data for the reports is calculated using record linkage with available administrative data sources. The reports are updated with new data 4 times each year and emailed directly to physicians who have requested their report. The reports are designed to have an overall dashboard summarizing the practice level data with a comparison to all physicians in Ontario. More detailed information on their data such as trend data and resources for quality improvement are found in subsequent pages of the report. These reports are a tool to support physicians in quality improvement efforts in their LTC practice. We believe the role of a medical director is very important in both the uptake and use of these reports as the medical director would act as a trusted advisor who can influence quality of care overall within an LTC home. We are also testing a new format for delivering the reports in an interactive online format that enables more options for viewing practice data. Initial evaluation of these reports shows that there is a statistically significant impact on reducing the prescription of antipsychotic medications in LTC homes. In future, we hope to see a larger effect on the latest topic included in the reports: antibiotic prescribing.